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Abstract 

The Next Generation Air Transportation System 
(NextGen) concept proposes many revolutionary 
operational concepts and technologies, such as 
display of traffic information and movements, airport 
moving maps (AMM), and proactive alerts of runway 
incursions and surface traffic conflicts, to deliver an 
overall increase in system capacity and safety. A 
piloted simulation study was conducted at the 
National Aeronautics and Space Administration 
(NASA) Langley Research Center to evaluate the 
ability to conduct safe and efficient airport surface 
operations while utilizing an AMM displaying traffic 
of various position accuracies as well as the effect of 
traffic position accuracy on airport conflict detection 
and resolution (CD&R) capability. Nominal 
scenarios and off-nominal conflict scenarios were 
conducted using 12 airline crews operating in a 
simulated Memphis International Airport terminal 
environment. The data suggest that all traffic should 
be shown on the airport moving map, whether 
qualified or unqualified, and conflict detection and 
resolution technologies provide significant safety 
benefits. Despite the presence of traffic information 
on the map, collisions or near collisions still 
occurred; when indications or alerts were generated 
in these same scenarios, the incidences were averted. 

Introduction 

The Next Generation Air Transportation System 
(NextGen) vision transforms the air transportation 
system to meet the projected growth in aircraft 
operations expected in the 2025 time-frame in a safe, 
efficient, and reliable manner [1]. NextGen is 
envisioned to remove many of the constraints in the 
current air transportation system, support a wider 
range of operations, and deliver an overall increase in 
system capacity and safety. Emerging NextGen 
operational concepts [2], such as trajectory-based 


airborne and surface operations, equivalent visual 
operations, and high-density arrival and departure 
operations, represent a revolutionary approach to air 
traffic management; as a result, a dramatic shift in the 
tasks, roles, and responsibilities for the flight deck 
are required to ensure a safe, sustainable air 
transportation system. A net- centric infrastructure 
will enable a safe and efficient airport surface 
environment by providing the information necessary 
to enable improved flight deck and ground vehicle 
situational awareness through display of traffic 
information and movements, airport moving maps 
(AMM), and proactive alerts of runway incursions 
and surface traffic conflicts. 

The National Aeronautics and Space 
Administration (NASA) is committed to developing 
technologies, data, and guidelines that may eliminate 
safety barriers in the terminal maneuvering area that 
would otherwise constrain the NextGen 
transformation. A Conflict Detection and Resolution 
(CD&R) concept has been developed that leverages 
advances in flight deck displays and technologies to 
promote enhanced surface and airborne traffic 
awareness with associated flight deck alerting 
concepts for safety assurance. These concepts employ 
continual ownship and traffic data monitoring and 
algorithms to detect potential conflicts on the runway 
and during taxi operations for aircraft and surface 
vehicles [3-10]. 

A high-fidelity full-motion piloted simulation 
study was conducted to evaluate the ability to 
conduct safe and efficient airport surface operations 
while utilizing an AMM displaying traffic of various 
position accuracies as well as the effect of traffic 
position accuracy on CD&R capability. The 
evaluation included approach, departure, and runway 
incursion scenarios, two AMM conditions, and traffic 
transmitting various levels of horizontal position 
accuracy. This paper describes the test set-up, 
method, and quantitative and qualitative results. 



System Description 

Simulator 

This research was conducted in the Research 
Flight Deck (RFD) simulator at NASA Langley 
Research Center (LaRC) (Figure 1) which is a high- 
fidelity, 6 degrees-of-freedom motion-based large 
commercial aircraft simulator with full-mission 
capability and advanced glass flight deck displays. 
Operations were conducted at the Memphis 
International (KMEM) airport. The out-the-window 
(OTW) scene included realistic taxiways and 
runways with appropriate markings, airport lighting, 
and other aircraft in simulated visibility conditions 
and provided approximately 20/40 visual acuity with 
a collimated 200° horizontal by 40° vertical field of 
view at 26 pixels per degree resolution. All standard 
audio call-outs were issued. 



Figure 1. RFD Flight Deck 

The RFD is equipped with dual commercial 
head-up displays; however, they were not utilized for 
this research. 

As shown in Figure 1, the simulator had four 
large main instrument panel displays referred to as: 
(left to right) pilot’s Primary Flight Display (PFD), 
pilot’s Navigation Display (ND), co-pilot’s ND, and 
co-pilot’s PFD. The four display panels were liquid 
crystal displays with 13.25 inch x 10.5 inch viewable 
area at 1280 x 1024 resolution. 

Two electronic flight bags (EFBs) were 
installed. Each provided a display resolution of 1 024 
X 768 pixels over a 10.4 inch diagonal area. The 
EFBs were mounted outboard of the PFDs and were 


used as the pilot’s interface for AMMs and charts. 
The EFBs were located within the pilot’s primary 
field-of-view as per the Federal Aviation 
Administration (FAA) Advisory Circular (AC) 25- 
11A[11]. 

Flight Deck Displays 

The PFD and ND were modeled after current 
state-of-the-art production aircraft. The ND was split, 
showing a half-screen navigational display and half- 
screen Engine Indication and Caution Alerting 
System. 

Electronic Flight Bag Display 

The EFB was used for the following functions: 
charts, checklists, and AMM. Test subjects interacted 
with the EFB through either the bezel buttons or a 
touch screen interface. 

The charts function provided the ability to 
display arrival, approach, departure, and airport 
charts. Pan/zoom (full page, half of the diagram, one 
fourth of the diagram) capability was provided. 

The checklists function provided the capability 
to display all aircraft checklists. 

The AMM was only displayed on the EFB. 
Range control for the AMM was provisioned at 0.4 
nautical miles (nm), 0.8 nm, 1.6 nm, and 3.2 nm. The 
basic AMM display included an airport layout 
showing runways, taxiways, and buildings. Surface 
(tan) and airborne (cyan) traffic icons were shown 
along with ownship position (white chevron). The 
AMM was generated using a KMEM airport 
geographic database developed to RTCA standards 
[12]. In this document, all figures of the AMM are 
shown without the surrounding EFB bezel button 
structure. Specific AMM configurations used for the 
study are provided in the Test Method section below. 

Air Traffic Control Simulation 

Air traffic control (ATC) instructions and pilot 
requests and replies were simulated to increase the 
fidelity of the terminal area environment simulation 
and provide normal pilot workload demands. All 
ATC and other aircraft radio messages were issued 
via a speech generation system. The messages were 
played through the intercommunication system or 



through the flight deck speaker system when the 
ownship and simulated traffic reached specified 
locations or timings to coincide with the scenario 
task. The subject pilots were asked to provide radio 
replies, when requested by the ATC messages, as per 
normal operating procedures. Human back-up was 
present in the event that additional or unscripted 
communication or clarification was necessary. 

Conflict Detection and Resolution 

A CD&R system was active during the testing. 
This system was developed based on Safety, 
Performance, and Interoperability Requirements 
(SPR) for Enhanced Traffic Situational Awareness on 
the Airport Surface with Indications and Alerts 
(SURF lA) specifications [13] as well as NASA 
CD&R research. SURF lA specifications were used 
to develop the runway conflict detection portion of 
the CD&R system. 

SURF lA 

SURF lA identifies potential runway conflicts 
that involve aircraft or vehicles in the airport 
maneuvering area, within 3 nm of the runway 
threshold and 1,000 ft above field elevation (AFE) 
and generates both indications and alerts for display 
to the flight crew. SURF lA utilizes traffic 
surveillance information obtained from Automatic 
Dependent Surveillance - Broadcast (ADS-B)-In and 
generates indications and alerts based on the 
aircraft/vehicle states during same runway, very 
closely spaced parallel runway, and intersecting 
runway operating configurations. Six types of aircraft 
operational states are defined: 1) taxiing on a taxiway 
toward a hold line or stopped at a hold line; 2) 
entering or crossing a runway (not lined up with 
runway); 3) takeoff; 4) approach; 5) after landing 
roll-out on runway (e.g., less than or equal to 40 kts); 
and, 6) stopped or taxiing along a runway. To prevent 
inappropriate crew responses during departure, 
indications and alerts are inhibited above 80 kts. The 
SURF lA application does not currently address 
taxiway or low altitude air-to-air conflicts, directive 
alerting, and is not intended for use on helicopters or 
vehicles. 

NASA CD&R Research 

NASA CD&R research has included 
development of algorithms to identify potential 
traffic conflicts at low altitudes near the airport, on 


the runway, and during taxi operations for multiple 
classes of aircraft and surface vehicles [14]. Since 
SURF lA does not currently include specifications 
for taxi conflicts, NASA research specific to taxi 
conflict detection was used to develop the taxi 
conflict detection portion of the CD&R system used 
for this testing. 

The NASA taxi conflict monitor was designed 
to detect and alert for ground taxi conflicts in the 
airport movement area by computing distances 
between the ownship and traffic, closing speeds, time 
to closest point of approach and other parameters to 
determine if criteria and thresholds are met for 
issuing alerts. Indications are not currently generated 
for taxi conflict situations. 

Indications and Alerts 

Indications and alerts (lAs) notified the flight 
crew of potentially hazardous situations and were 
presented on the AMM. lA specifications are defined 
in [13]. 

Indications were intended to generate pilot 
awareness and situation assessment by highlighting 
the runway and traffic status as relevant to ownship 
operations. Indications identified operational 
conditions that were generally normal, yet relevant 
for runway safety and could be a precursor to a non- 
normal situation. Only visual annunciations were 
required and used for indications. Indications were 
only issued for runway conflict situations. Two types 
of indications were utilized. 

A traffic indication (TI) highlighted a potential 
runway traffic collision/hazard that could emerge in 
the near future. TIs were intended to increase the 
flight crews’ awareness of the relevant runway 
traffic. The flight crew could proceed with the 
intended operation after a brief assessment of the 
situation and if appropriately cleared. A TI was 
displayed on the AMM as an enlarged traffic symbol 
for the relevant traffic surrounded by a dashed circle 
in the same color as the traffic symbol and an 
identification tag that showed flight identification and 
ground speed in knots. A status message (“Traffic”) 
was displayed at the bottom of the AMM along with 
the estimated distance to the traffic in nautical miles 
until below 0.1 nm (600 ft), then displayed in feet. 

A runway status indication (RSI) identified 
whether the runway that the ownship was 



approaching or using was in-use or occupied by other 
traffic and was not suitable for entering, takeoff, or 
landing. Before proceeding, the crew should ensure 
they had the appropriate clearance and the indicated 
traffic was not a factor. An RSI was displayed on the 
AMM (Figure 2) in the same manner as a TI with the 
addition of a solid blue line outlining the relevant 
runway. 



Figure 2, AMM Showing Runway Status 
Indication 

Alerts identified potential collision hazards 
which require immediate awareness by the flight 
crew and may require timely action or response to 
avoid a collision. Alerts had priority over indications. 
Auditory and visual annunciations were required for 
alerts. Alerts were issued for both runway and taxi 
conflict situations. A two-level alerting scheme was 
utilized. 

Caution alerts were generated for conditions 
that required immediate flight crew awareness and 
subsequent flight crew response. Generally, caution 
alerts were issued with sufficient time to allow a crew 
to evaluate the situation and to prepare to respond. A 
caution alert was displayed on the AMM as an 
enlarged yellow traffic symbol surrounded by a 
yellow circle for the relevant traffic, an identification 
tag that showed flight identification and ground speed 
in knots, and a yellow line around the relevant 
runway, if applicable. An alert message (“Caution, 


Traffic”) was displayed at the bottom of the AMM in 
yellow text along with the estimated distance to the 
traffic. An audible annunciation was also made 
(“Caution, Traffic, Caution, Traffic”). 



Figure 3. AMM Showing Warning Alert 

Warning alerts were issued for conditions that 
required immediate flight crew awareness and 
immediate flight crew response. Warning alerts could 
occur without preceding caution alerts. A warning 
alert was displayed in the same manner on the AMM 
as a caution alert, except the warning was associated 
with the color red, a square was used to surround the 
traffic symbol, and the alert message was “Warning, 
Traffic” (Figure 3). 

Surveillance Data 

The quality and accuracy of reported traffic 
surveillance data are critical to the integrity of the 
AMM traffic displays and the CD&R capability. The 
traffic position accuracy was simulated as dependent 
upon the Global Positioning System (GPS) 
measurement errors. A Gauss-Markov process 
modeled the time correlation between successive 
position measurement errors [15]. It was assumed 
that ADS-B would be used as the means for 
transmitting (ADS-B Out) and receiving (ADS-B In) 
these GPS-based aircraft surveillance data. Although 
ADS-B transmission qualities and effects were not 
modeled for this study, the traffic positional data was 



updated at one hertz to simulate ADS-B transmission 
rates. Latency effects, transmission line-of-sight, 
bandwidth blockage, and vertical position accuracy 
were not modeled. 

The Navigation Accuracy Category for Position 
(NACp) describes the accuracy of positional 
information. NACp values range from 0 to 11 [16]. 
The NACp categories of 8 and higher are listed in 
Table 1 with their associated horizontal Estimated 
Position Uncertainty (EPU) values. 


Table 1. NACp Horizontal Accuracy Bound 


NACp 

95% Horizontal Accuracy - Estimated 
Position Uncertainty (EPU) 

8 

EPU < 92.6 m (0.05 nm, 305.6 ft) 

9 

EPU < 30 m (99 ft) 

10 

EPU < 10 m (33 ft) 

11 

EPU < 3 m (9.9 ft) 


The FAA has issued an ADS-B Out Final Rule 
[17] which includes performance standards. The rule 
states that EPU of the reported position must be less 
than 0.05 nm, which is equivalent to NACp 8. 

Both DO-289 (Minimum Aviation System 
Performance Standards for Aircraft Surveillance 
Applications) [18] and DO-322 (SPR for Airborne 
Traffic Situational Awareness for Surface Operations 
(ATSA-SURF) Application) [19] define the 
minimum requirement for horizontal position 
accuracy for depiction of ground traffic on an AMM 
to be at least 30 m within 95% containment bounds, 
equivalent to NACp 9. Requirements for other 
navigation parameters are also specified in these 
documents but will not be addressed in this paper. 
The positional surveillance accuracy of NACp 8, as 
specified in the ADS-B Out rule, is not sufficient to 
support the depiction of traffic information on an 
AMM display, per DO-322, although possible 
mitigation strategies have been proposed (e.g., RTCA 
DO-322, Appendix D) [19]. 

The SURF lA SPR has proposed horizontal 
position accuracy requirements [13] for the SURF lA 
function. Through analysis, the SPR identified that to 
meet safety requirements, horizontal position 
accuracy when on the airport surface needs to be at 
least 10 m within 95% containment bounds (NACp 
10) to allow indications and alerts to be issued for 


traffic at virtually all airports in the National 
Airspace System. Validation of these requirements is 
on-going. 

To span the ADS-B Out Final Rule, ATSA- 
SURF, and SURF lA position accuracy reporting 
requirements (Table 2), traffic position accuracies 
equivalent to NACp 8, 9, 10, and 1 1 levels were used 
for this test. Truth data, with no accuracy errors, was 
also recorded. The capability was provided to enable 
selection of the desired accuracy level (e.g. NACp 8, 
NACp 9, etc.) for simulated traffic individually. For 
this study, traffic transmitting horizontal position 
accuracy equivalent to NACp 8 definitions was 
considered unqualified traffic. Traffic transmitting 
position accuracy equivalent to NACp 9 and higher 
was considered qualified traffic. 


Table 2. NACp Accuracy Requirements 


NACp 

ADS-B Out 
Final Rule 

Display Per 
DO-322 
(ATSA-SURF) 

I As Per 
DO-323 
(SURF lA) 

8 

Yes 

No 

No 

9 

Yes 

Yes 

No 

10 

Yes 

Yes 

Yes 

11 

Yes 

Yes 

Yes 


Test Method 

The objectives of the testing were to evaluate the 
ability of a subject crew to conduct safe and efficient 
airport surface operations while utilizing an AMM 
displaying traffic of various position accuracies as 
well as the effect of traffic position accuracy on 
CD&R capability. The evaluation included approach, 
departure, and runway incursion scenarios and two 
AMM conditions with traffic transmitting various 
levels of horizontal position accuracy. 

Evaluation Pilots 

Twenty-four commercial pilots served as 
participants for the research. The test subjects were 
paired by airline and role (Captain, First Officer) to 
ensure crew coordination and cohesion with regard to 
terminal and surface standard operational procedures. 
All pilots held an Airline Transport Pilot rating. The 
Captains had an average of over 17,000 flight hours 
with 25 years of commercial experience. The First 
Officers had an average of over 13,000 flight hours 
with 20 years of commercial experience. 





AMM Display Conditions 

These AMM eonditions were ehosen to evaluate 
the effects of displaying all airport traffic on an 
AMM (Map B) versus displaying only qualified 
traffic on an AMM (Map A). 

The Map A condition consisted of the basic 
AMM format displaying qualified traffic only (traffic 
reporting a horizontal position accuracy of NACp 9 
and higher). As a result, the locations of some of the 
airport traffic (those reporting horizontal position 
accuracy of NACp 8) were not displayed. The only 
method of acquiring the NACp 8 reporting traffic 
was visually, OTW. 



Figure 4. AMM Display Showing All Traffic 

The Map B condition consisted of the basic 
AMM format displaying both qualified and 
unqualified traffic (traffic reporting a horizontal 
position accuracy of NACp 8) (Figure 4); therefore, 
all airport traffic was displayed on the AMM. As 
position accuracy decreased, the location of traffic 
symbols could vary from the traffic’s actual location. 
For example, the traffic symbol for an aircraft 
transmitting a position accuracy of NACp 8 could be 
displayed 305 ft or more from the aircraft’s actual 
location, giving a potentially misleading indication of 
the traffic’s location. 

The NACp levels of the traffic position reports 
were not indicated to the crew by icon shape or color 
on the AMM concepts 


Test Matrix 

A total of 10 trials (8 nominal, 2 off-nominal) 
were conducted by each crew. All trials were 
conducted in daytime 1,800 ft visibility. 

Nominal trials were conducted to evaluate the 
effect of displaying traffic transmitting various 
position accuracies on the AMM during airport 
surface operations. Traffic was transmitting position 
accuracy ranges from NACp 8 to NACp 11. Two 
AMM conditions (Map A and Map B) were 
evaluated across subjects during two approach and 
two departure scenarios, for a total of eight nominal 
trials. 

For the approach scenarios, the ownship was 
initialized approximately 5 nm from the runway 
threshold for a standard auto-land approach. After 
landing and roll-out, the aircraft was to exit the 
runway and taxi to the ramp via the designated taxi 
route. For the departure scenarios, the subject crew 
began taxi from the ramp, taxied to the departure 
runway via the designated taxi route, and executed a 
takeoff after receiving the appropriate clearance. The 
trial terminated once the aircraft reached 
approximately 1000 ft AFE. The pilots were 
requested to maintain a standard taxi speed of 15 kts 
whenever feasible. 

Off-nominal trials were conducted between 
subjects to evaluate the potential safety implications 
of traffic position accuracy on airport surface CD&R 
capability. Each crew completed two off-nominal 
trials as Test Runs 6 and 10 of 10 trials. 

For six of the crews, the objective was to 
evaluate the impact of displaying qualified versus 
unqualified traffic on the AMM during conflict 
situations. These crews were given a taxi crossing 
and departure conflict scenario (see below for 
description), using either Map A or B display 
condition. The conflict traffic was transmitting NACp 
8 position accuracy for all these trials; therefore, lAs 
were not issued. When using Map A, the conflict 
traffic was not displayed on the AMM and only could 
be viewed OTW. When using Map B, the conflict 
traffic was displayed on the AMM, albeit with NACp 
of 8 accuracy, and could also be viewed OTW. 

For the other six crews, the objective of the off- 
nominal trials was to evaluate the impact of receiving 



versus not receiving lAs for traffic displayed on the 
AMM during conflict situations. These crews were 
given a taxi crossing and departure conflict scenario 
using the Map A display condition. The conflict 
traffic was transmitting either NACp 9 or NACp 10 
position accuracy and, therefore, was always 
displayed on the AMM. lAs were not issued for 
traffic transmitting NACp 9 position accuracy, but 
were issued for traffic transmitting NACp 1 0 position 
accuracy. 

Conflict Scenarios 

Two conflict scenarios were utilized. Every 
effort was made to produce similar timings; however, 
a certain amount of variability was naturally 
introduced due to the maneuvering conducted by the 
pilot (e.g., taxi speed). 

Taxi Crossing Conflict Scenario 

An approach and departure flow was simulated. 
Traffic was approaching Runway 36C (Figure 5, 
green route) and Runway 36L (Figure 5, orange 
route), spaced 5 nm apart and staggered between 
runways. This traffic landed, exited the runway, and 
taxied to the terminal via the routes shown in Figure 
5. There was a departure flow using Runway 36R via 
the blue colored route shown in Figure 5. Other static 
traffic was placed in strategic locations to add interest 
to the scenario. 

At the beginning of this scenario, the ownship 
was parked on the ramp near Taxiway M6 facing 
Taxiway N. The flight crew was cleared to taxi to 
Runway 36C via Taxiways N, P, S, and R (Figure 5, 
magenta route), holding short of Runway 36C. As the 
ownship approached the Runway 36C hold line, an 
aircraft was landing on the runway. The subject crew 
was cleared to cross Runway 36C when the landing 
traffic was approximately 8,000 ft away. 

This scenario tested the incursion situation 
where an aircraft taxis across a runway even though 
another aircraft is landing on the same runway. In 
this case, the subject crew was mistakenly cleared to 
cross the runway, resulting in a potential collision 
unless action was taken. 

Departure Conflict Scenario 

A southern approach and departure flow was 
simulated. Traffic was approaching Runway 18R 



Figure 5. Taxi Crossing Conflict Scenario 

(Figure 6, orange route) and Runway 18F (Figure 6, 
blue route), spaced 5 nm apart and staggered between 
runways. This traffic landed, exited the runway, and 
taxied to the terminal via the routes shown in Figure 
6. There was a departure flow using Runway 18C via 
the magenta colored route shown in Figure 6. 

One aircraft taxied from the ramp via the green 
route shown in Figure 6 and held short of Runway 
18C at Taxiway D. This aircraft was intended to be a 
distraction for the subject crew. Other static traffic 
was placed in strategic locations to add interest to the 
scenario. 

At the beginning of this scenario, the ownship 
was parked on the ramp facing Taxiway J. The flight 
crew was cleared to taxi to Runway 18C via 
Taxiways J, K, C, and C8 (Figure 6, magenta route), 
holding short of Runway 18C. As the ownship was 
cleared and was taxied onto the runway, another 
aircraft taxied from the ramp (Figure 6, yellow route) 
and held short of Runway 18C at Taxiway F. As the 
ownship began its departure roll, the aircraft holding 
on Taxiway F crossed the runway in front of the 
ownship. Taxiway F is 4,500 ft from the Runway 
18C threshold. 

This scenario tested the incursion situation 
where an aircraft taxis across a runway in front of a 
departing aircraft. The traffic for this scenario. 


instead of holding short, taxied aeross Runway 18C 
without elearanee as the ownship began its departure. 



Figure 6. Departure Conflict Scenario 
Procedure 

Prior to the testing phase, eaeh test subjeet 
participated in a briefing and training session. The 
training included general simulator characteristics 
and specific items and procedures related to the 
testing, such as Map Conditions A and B, traffic 
position accuracy, and CD&R capability. 

During the training for the CD&R capability, the 
flight crews were trained to abort if a warning alert 
was generated during departure, go-around if a 
warning alert was generated on approach, and stop if 
a warning alert was generated during taxi. They were 
not required to take evasive action when an 
indication or caution alert was issued. 

Before each trial, the flight crew was briefed on 
the run conditions, e.g., approach or departure, 
visibility, winds, display condition in use, and ATC 
radio frequencies. 

The test runs were documented via audio, video, 
and digital data recordings, and post-run and post-test 
questionnaires. 


Test Metrics 

For conflicts, a near collision was counted if the 
center-of-gravity (CG) of the two aircraft were <300 
ft apart laterally and vertical separation was < 200 ft. 
A collision was counted if the aircraft CGs were < 
150 ft apart laterally and vertical separation was < 
100 ft. 

Results 

A summary of quantitative and qualitative 
results is presented. All data are referenced from the 
aircraft CG. All statistically significant results are 
presented at the p < 0.05 level. Nominal scenario 
results are included in the qualitative results section. 

Off-Nominal Scenario Results 

As described in the Test Method section, the test 
objective for six of the crews was to evaluate the 
impact of displaying qualified versus unqualified 
traffic on the AMM during conflict situations. These 
crews were given a taxi crossing and departure 
conflict scenario, using either Map A or B display 
condition. The conflict traffic was transmitting NACp 
8 position accuracy for all of these trials; therefore, 
lAs were not issued. When using Map A, the conflict 
traffic was not displayed on the AMM and could only 
be viewed OTW, if possible for the visibility 
conditions. When using Map B, the conflict traffic 
was displayed on the AMM and could also be viewed 
OTW. 

When using the Map A condition, action was 
only taken to avoid the conflict traffic on one of six 
trials (17%) (Table 3). The flight crew was not aware 
of the conflict traffic on five of these trials and 
continued the operations, resulting in four collisions 
and one near collision. On one trial, after seeing the 
traffic OTW on takeoff roll, a high speed rejected 
takeoff (131 kts) was conducted. The aircraft stopped 
just before reaching the traffic, resulting in a near 
collision. 

When using the Map B condition, action was 
taken on four of six trials (67%) (Table 3). For the 
taxi crossing scenario, one crew crossed the runway, 
unaware of the conflict traffic, resulting in a 
collision. This crew viewed the AMM for traffic, 
zooming out to the largest map scale; the map was 


scanned just before the traffic came into view. For 
the departure scenario, the crew that departed saw the 
traffic on the AMM but was not sure if the traffic was 
on the runway due to the traffic’s position accuracy 
(aircraft symbol was “dancing” on the AMM). 


Table 3. Off-Nominal Scenario Results, Map 
Condition Focus 



Taxi Crossing 

Departure 

Map A, 
NACp 8 

3 crossed: 

2 collisions, 

1 near collision 

2 departures, 

1 high speed reject: 

2 collisions, 

1 near collision 

Map B, 
NACp 8 

1 crossed, 

2 held short: 

1 collision 

1 departure, 

2 held in position: 

1 collision 


For the other six crews, the objective of the off- 
nominal trials was to evaluate the impact of receiving 
lAs versus not receiving them for traffic displayed on 
the AMM during conflict situations. These crews 
were given a taxi crossing and departure conflict 
scenario using the Map A display condition. The 
conflict traffic was transmitting either NACp 9 (no 
lAs) or NACp 10 (lAs issued) position accuracy and, 
therefore, was always displayed on the AMM. 

When the conflict traffic was transmitting NACp 

9 accuracy, no action was taken on three of six trials 
(50%) (Table 4). Two of the taxi crossing trials 
resulted in collisions; the crews did not see the traffic 
on the AMM and crossed the runway. Two of the 
departure trials also resulted in collisions. During one 
of the departure trials, the crew saw the conflict 
traffic on the AMM but were not sure if the traffic 
was on the runway due to the traffic’s position 
accuracy so they continued the departure. During the 
other departure trial, the crew saw the traffic OTW 
and conducted a high speed rejected takeoff (132 
kts), having to veer to the right of the conflict traffic, 
which resulted in a collision. 

When the conflict traffic was transmitting NACp 

10 accuracy, action was taken to avoid the conflict on 
all six trials (Table 4). For the three taxi crossing 
trials, action was based on receiving RSIs. For the 
three departure trials, action was based on receiving a 
warning, an RSI, and viewing the traffic on the 
AMM. 


Table 4. Off-Nominal Scenario Results, NACp 
Focus 



Taxi Crossing 

Departure 

Map A, 
NACp 9 

2 crossed, 

1 held short: 

2 collisions 

1 departure, 2 reject: 

2 collisions 

Map A, 
NACp 10 

3 held short 

1 held in position, 

2 rejected 


Taxi Crossing Off-Nominal Details 

The details of the taxi crossing off- 
nominaFconflict trials are presented in Table 5. 

The data shown that, with all traffic displayed 
and with an increase in position accuracy, the number 
of collisions/near collisions were reduced. All trials 
in which the conflict traffic was not displayed on the 
AMM (Map A condition with conflict traffic 
transmitting NACP 8 position accuracy) resulted in a 
collision or near collision. Collisions/near collisions 
were reduced when all the traffic was displayed on 
the AMM (Map B condition and Map A condition 
with conflict traffic transmitting NACp 9 accuracy). 
When lAs were issued (conflict traffic transmitting 
NACp 10 accuracy), there were no collisions or near 
collisions. 

In Table 5, all distances are referenced to the CG 
(the CG of the ownship was 72.8 ft from the nose of 
the aircraft). For all trials but one in which the 
aircraft held short of the hold line (HL), the nose was 
well behind the hold line (40 to 98 ft). In this one 
case, the nose did cross the hold line (technically a 
runway incursion) but stopped -150 ft before 
reaching the runway edge. 

lAs were possible on the conflict traffic on three 
taxi crossing trials (transmitting NACp 10 accuracy). 
A TI was generated on one trial, ~3.4 nm from the 
traffic, 212 ft prior to crossing the hold line. An RSI 
was generated on all three trials, 10,341 ft mean 
distance from the traffic (standard deviation (a) = 
7,373 ft), 206 ft mean distance (a = 13.5 ft) prior to 
crossing the hold line. The large standard deviation 
was caused by the wide variability in taxi operations. 
One crew taxied slower than the planned 15 kts and 
reached the runway hold line when the traffic was 
only 2,586 ft from the ownship. In contrast, another 
crew generally taxied faster than the planned 15 kts 



and reached the runway hold line before conditions 
were met for lAs (TI and RSI later issued). 

lAs were also generated for other traffic during 
the off-nominal trials. TIs were generated on all 12 
taxi crossing trials for an aircraft that was landing 
behind the conflict traffic as the ownship was 
crossing the runway (distance from traffic, mean (p) 
= 35,222 ft, a = 74 ft). One crew received an RSI on 
this same aircraft (distance from traffic, 17,147 ft). 
Three crews also received lAs on an aircraft parked 


at a gate in the ramp area (one caution, 49 ft from 
traffic; two warnings, p = 265 ft, a = 1.7 ft from 
traffic). Even though the static traffic was 
transmitting NACp 10 position accuracy, for these 
trials, there was a larger error on the position data as 
calculated by the surveillance data model, placing the 
aircraft closer to the path of the ownship. Since 
aircraft can be closer together in the congested ramp 
area, perhaps lAs should be inhibited in the ramp 
area to minimize nuisance alerting caused by position 
data inaccuracies. 


Table 5. Taxi Crossing Scenario Results 


Map 

NACp 

Action 

Outcome 

CPA* 

(feet) 

CG Dist to HL (feet) 

Traffic Awareness 

A 

8 

Crossed Runway 


21.3 

N/A 

Not Aware 

A 

8 

Crossed Runway 

Near Collision 

262.9 

N/A 

Not Aware 

A 

8 

Crossed Runway 


104.1 

N/A 

Not Aware 

B 

8 

Held Short 

Stopped 

385.3 

134.5 

AMM 

B 

8 

Held Short 

Stopped 

452.9 

114.6 

AMM 

B 

8 

Crossed Runway 


154.4 

N/A 

Not Aware - early 
map check 

A 

9 

Held Short 

Sto) 

U! 

•ed 

400.1 

156.5 

AMM 

A 

9 

Crossed Runway 




161.9 

N/A 

Not Aware 

A 

9 

Crossed Runway 




63.9 

N/A 

Not Aware 

A 

10 

Crossed HL 

Stopped 

299.1 

19.0 

RSI 

A 

10 

Held Short 

Stopped 

398.2 

113.6 

RSI 

A 

10 

Held Short 

Stopped 

471.4 

171.1 

RSI 


* CPA - Closest Point of Approach 


Table 6. Departure Scenario Results 


Map 

NACp 

Action 

Outcome 

CPA (feet) 

RTO GS* 

Traffic Awareness 

A 

8 

Took-off 


44.0 

N/A 

Not Aware 

A 

8 

TO roll 

RTO - Near 
Collision 

209.9 

131 

OTW 

A 

8 

Took-off 


151.3 

N/A 

Not Aware 

B 

8 

Held Position 

Stopped 

4,054.4 

N/A 

AMM 

B 

8 

Held Position 

Stopped 

4,214.1 

N/A 

AMM 

B 

8 

Took-off 


74.6 

N/A 

Map but unsure due to 
accuracy issue 

A 

9 

Start TO 

RTO 

2,626.5 

35 

AMM 

A 

9 

TO roll 


169.4 

132 

OTW 

A 

9 

Took-off 


128.3 

N/A 

Map but unsure due to 
accuracy issue 

A 

10 

Initial roll 

Stopped 

4,240.8 

N/A 

AMM 

A 

10 

Start TO 

RTO- Stopped 

1,386.8 

93 

Warning 

A 

10 

Start TO 

RTO- Stopped 

3,390.9 

47 

RSI, but got Warning 


*RTO GS - Roll-out Turn-Off Ground Speed 



Departure Off-Nominal Details 

The details of the departure conflict trials are 
presented in Table 6. 

As with the taxi crossing trials, the number of 
collisions/near collisions were reduced when all 
traffic was displayed and position accuracy was 
increased. All trials in which the conflict traffic was 
not displayed on the AMM (Map A condition with 
conflict traffic transmitting NACP 8 position 
accuracy) resulted in a collision or near collision. 
Collisions/near collisions were reduced when all the 
traffic was displayed on the AMM (Map B condition 
and Map A condition with conflict traffic 
transmitting NACp 9 accuracy). However, when lAs 
were issued (traffic transmitting NACp 10 accuracy), 
there were no collisions or near collisions. The CPA 
distance varied greatly depending on the action of the 
pilot. Also, in all trials in which the aircraft took-off, 
the collision occurred just after liftoff when the 
aircraft was 40 ft or less above ground level (AGL). 

Based on these departure test trials, the pilots 
rejected the takeoff sooner when traffic awareness 
occurred based on either viewing the conflict traffic 
on the AMM or when lAs were issued. By the time 
the conflict traffic could be viewed OTW, it was too 
late to conduct a successful high speed rejected 
takeoff (RTO) resulting in a collision and near 
collision. 

lAs were possible on the conflict traffic on three 
departure trials (transmitting NACp 10 accuracy). An 
RSI was generated on one trial prior to receiving a 
warning alert, 4,268 ft from the traffic, when 
traveling 21 kts. A warning alert was generated on 
two trials, 3,898 ft mean distance from the traffic (a = 
298 ft), when traveling 51 kts mean (a = 15 kts). 
After initiating an RTO/stop, an RSI was generated 
on all three departure trials (distance from traffic, p = 
3,338 ft, a = 1,043 ft; ground speed, p = 25 kts, a = 
16 kts). 

Qualitative Results 

Post-run and post-test questionnaires were 
administered. Most of the questions were rated on a 
scale of 1 (“strongly disagree”, “low”, “not usefuf’) 
to 7 (“strongly agree”, “high”, “very useful”). 


Traffic Awareness and Symbology - Based on 
post-run responses, the map condition showing all 
traffic (Map B: p = 5.4, a = 2.1) was significantly 
better for traffic awareness than the map condition 
only showing qualified traffic (Map A: p = 3.0, a = 

2.1) . Furthermore, the pilots indicated the 

presentation of only qualified traffic on the AMM 
was a greater safety issue (p = 6.3, a = 1.1) than 
displaying all traffic (p = 2.3, a = 1.7). They also felt 
the display of only qualified traffic on the AMM will 
increase the potential for accidents (p = 5.5, a = 1.4) 
than the display of all traffic (p = 2.0, a = 1.6). When 
asked if all traffic should be displayed on the AMM 
or only qualified traffic (transmitting NACp 9 and 
higher position accuracy levels), 23 of 24 pilots 
(96%) responded that all traffic should be displayed 
(confidence level of p = 6.4, a = 0.6, where 7 = 100% 
confidence). These results are consistent with the 
quantitative performance that shows fewer accidents 
and incidents with all traffic being displayed. 

As one would expect, when the displayed traffic 
was of lower position accuracy (NACp 8 condition: p 
= 3.0, a = 2.1 vs. NACp 10 condition : p = 6.1, a = 

1.1) , it was rated significantly lower for traffic 
awareness. The pilots were asked if the NACp 
surveillance accuracy should be shown on the AMM. 
The pilot ratings indicated a slight preference for 
using different symbology for qualified and 
unqualified traffic (p = 5.4, a = 2.0) instead of using 
the same symbology (p = 4.5, a = 1.7) to avoid 
potentially misleading information (not statistically 
significant). 

For this study, the AMM was located on the 
EFB outboard of the PFD. The pilots moderately 
agreed that this was an optimal location for viewing 
the AMM (p = 5.3, a = 1.4). Several pilots 
commented that the AMM should be located on a 
forward display, preferably the ND, or in a central 
location for viewing by both pilots. 

Conflict Awareness and CD&R Symbology - 
The possibility exists that the flight crew may not be 
aware of potential conflict traffic when using Map A 
condition (showing only qualified traffic), 
particularly if visibility is low and traffic cannot be 
visually acquired OTW. Based on the post-run 
responses, the map condition showing all traffic 
(Map B: p = 5.3, a = 1.9) was significantly better for 
detection of potential surface conflicts than the map 



condition only showing qualified traffic (NACp 9 and 
higher) (Map A: p = 1.3, a = 0.5). 

When the conflict traffic was transmitting NACp 
8 position accuracy, the traffic was not displayed on 
the AMM. The conflict traffic was displayed on the 
AMM when it was transmitting NACp 9 accuracy; 
however, lAs were not issued. When transmitting 
NACp 10 accuracy, the conflict traffic was displayed 
on the AMM and lAs were issued, providing the 
most information related to potential surface 
conflicts. Traffic transmitting more accurate position 
data (NACp 9: p = 3.9, a = 2.2 and NACp 10: p = 
6.7, a = 0.5) was rated higher for detection of 
potential surface conflicts than traffic transmitting 
less accurate data (NACp 8: p = 1.3, a = 0.5). 

The pilots rated the CD&R symbology as 
effective in providing information on the conflict 
traffic (p = 6.4, a = 1.0) and providing a clear 
indication of the relative location of the conflict 
traffic (p = 6.5, a = 0.7). Both indications and alerts 
were helpful in determining critical runway safety 
information (indications: p = 6.3, o = 1.2, alerts: p = 
6.5, a = 0.6), provided additional information over 
AMM traffic (indications: p = 6.2, a = 0.9, alerts: p = 
6.4, a = 0.7), and helped in determining the location 
and movement of traffic that was relevant to the 
safety of their own aircraft (indications: p = 6.3, a = 
0.8, alerts: p = 6.4, a = 0.7). 

The pilots felt there should be a distinction 
(difference) between the symbology representing 
traffic qualified for the CD&R indication and alerting 
function (traffic transmitting NACp 10 or 1 1) versus 
traffic not qualified for that function (traffic 
transmitting NACp 9 or less) (p = 5.3, a = 1.7). 

General - The pilots rated their level of 
perceived safety higher having the CD&R system 
onboard their aircraft during similar operations (p = 
6.2, a = 0.6) than without having the system onboard 
(p = 4.8, a = 0.9) (statistically significant). With a 
rating of 7 being completely safe, the pilots rated the 
level of safety felt during runway conflict incidents 
(p = 5.7, a = 1.0) and during taxi conflict incidents (p 
= 5.9, a = 0.8) as moderately high. The pilots were 
asked to provide suggestions for improvements that 
would increase the safety of the system. Some of the 
prevalent comments were: more experience with the 
system; detailed training, including appropriate map 
scale per operation; reduce number of menu levels 


(button pushing) on the EFB; display all traffic on the 
AMM; provide lAs for all traffic; and directive 
alerting. Also, the question of qualified vs. 
unqualified traffic and traffic position accuracy added 
to mental workload and uncertainty. Some of the best 
features identified were the display of traffic on the 
AMM and lAs. 

Summary 

A piloted simulation study was conducted to 
evaluate the ability to conduct safe and efficient 
airport surface operations while utilizing an AMM 
displaying traffic of various position accuracies as 
well as the effect of traffic position accuracy on 
airport CD&R capability. 

With all traffic displayed and with an increase in 
position accuracy, the number of collisions/near 
collisions was reduced. A collision or near collision 
occurred on each trial in which the conflict traffic 
was not displayed on the AMM (transmitting NACp 
8 accuracy). There were no collisions when lAs were 
generated on the conflict traffic (transmitting NACp 
> 10 accuracy), notifying the flight crew of the 
potential conflict situation. 

The test subjects thought that all the traffic 
should be displayed on the AMM, regardless of the 
position accuracy level, to provide a higher level of 
traffic awareness, increased safety, and to reduce the 
potential for accidents. There are some concerns with 
displaying traffic transmitting less accurate NACp 8 
position accuracy, however. Due to the level of 
position uncertainty, pilots were sometimes unsure if 
the traffic was actually on the runway and continued 
the operation, resulting in collision. 

The CD&R system was very effective in 
preventing collisions; no collisions occurred when 
lAs were generated for conflicting traffic. The 
current design, however, issued nuisance alerts on 
traffic in the ramp area due to position accuracy, even 
at the NACp 1 0 level. 
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